On Geomatric and Algebraic Aspects of 3D Affine and Projective Structures from Perspective 2D Views

Part I of this paper investigates the diierences | conceptually and algorithmically | between aane and projective frameworks for the tasks of visual recognition and reconstruction from perspective views. It is shown that an aane invariant exists between any view and a xed view chosen as a reference view. This implies that for tasks for which a reference view can be chosen, such as in alignment schemes for visual recognition, projective invariants are not really necessary. The projective extension is then derived, showing that it is necessary only for tasks for which a reference view is not available | such as happens when updating scene structure from a moving stereo rig. The geometric diierence between the two proposed invariants are that the aane invariant measures the relative deviation from a single reference plane, whereas the projective invariant measures the relative deviation from two reference planes. The aane invariant can be computed from three corresponding points and a fourth point for setting a scale; the projective invariant can be computed from four corresponding points and a fth point for setting a scale. Both the aane and projective invariants are shown to be recovered by remarkably simple and linear methods. In part II we use the aane invariant to derive new algebraic connections between perspective views. It is shown that three perspective views of an object are connected by certain algebraic functions of image coordinates alone (no structure or camera geometry needs to be involved). In the general case, three views satisfy a trilinear function of image coordinates. In case where two of the views are orthographic and the third is perspective the function reduces to a bilinear form. In case all three views are orthographic the function reduces further to a linear form (the \linear combination of views" of 31]). These functions are shown to be useful for recognition, among other applications.